Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion
نویسندگان
چکیده
منابع مشابه
Conditional Modality Fusion for Coreference Resolution
Non-verbal modalities such as gesture can improve processing of spontaneous spoken language. For example, similar hand gestures tend to predict semantic similarity, so features that quantify gestural similarity can improve semantic tasks such as coreference resolution. However, not all hand movements are informative gestures; psychological research has shown that speakers are more likely to ges...
متن کاملDeMIAN: Deep Modality Invariant Adversarial Network
Obtaining common representations from different modalities is important in that they are interchangeable with each other in a classification problem. For example, we can train a classifier on image features in the common representations and apply it to the testing of the text features in the representations. Existing multi-modal representation learning methods mainly aim to extract rich informa...
متن کاملAn Exploration of the Impacts of Three Factors in Multimodal Biometric Score Fusion: Score Modality, Recognition Method, and Fusion Process
Operational applications for human identification require high credibility in order to determine or verify a person’s identity to a desired confidence level. Multimodal biometric score fusion (MBSF) can significantly improve detection, recognition, and identification performance of humans. The goals of this research are to explore the impact of each factor in a MBSF process and to determine the...
متن کاملIs susceptibility to perceptual migration and fusion modality-specific or multimodal?
A previous paper reported high susceptibility to spatial migration (allochiria) of tactile stimuli in about 25% of healthy individuals (High Error subjects). When synchronous stimuli touched the two hands, if the unattended stimulus was temporally modulated when the attended one was not (and was thus more salient than the latter), it "migrated" to and fused with or replaced the stimulus on the ...
متن کاملModality Fusion in a Route Navigation System
In this paper we present the MUMS Multimodal Route Navigation System which combines speech, pen, and graphics into a PDA-based multimodal system. We focus especially on the three-level modality fusion component which we believe provides an accurate and more flexible input fusion than the usual twolevel approaches. The modular architecture of the system supports flexible component management, an...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the AAAI Conference on Artificial Intelligence
سال: 2020
ISSN: 2374-3468,2159-5399
DOI: 10.1609/aaai.v34i01.5347